Generating Classifier Commitees by Stochastically Selecting both Attributes and Training Examples

Boosting and Bagging, as two representative approaches to learning classiier committees, have demonstrated great success, especially for decision tree learning. They repeatedly build diierent classiiers using a base learning algorithm by changing the distribution of the training set. Sasc, as a diierent type of committee learning method, can also signiicantly reduce the error rate of decision trees. It generates classiier committees by stochastically modifying the set of attributes but keeping the distribution of the training set unchanged. It has been shown that Bagging and Sasc are, on average, less accurate than Boosting, but the performance of the former is more stable than that of the latter in terms of less frequently obtaining signiicantly higher error rates than the base learning algorithm. In this paper, we propose a novel committee learning algorithm, called SascBag, that combines Sasc and Bagging. It creates diierent classiiers by stochastically varying both the attribute set and the distribution of the training set. Experimental results in a representative collection of natural domains show that, for decision tree learning, the new algorithm is, on average, more accurate than Boosting, Bagging, and Sasc. It is more stable than Boosting. In addition, like Bagging and Sasc, SascBag is amenable to parallel and distributed processing while Boosting is not. This gives SascBag another advantage over Boosting for parallel machine learning and datamining.